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FIG.6 

1 :<smil> 
2: <head> 

3: <meta name="title" content="navigation" /> 
4: <meta name="author" content="Okure" /> 
5: <meta name="role" content="template" /> 
6: <layout> 

7: <root-layout width="352" height="144"/> 

8: <region id="Image" width="176" height="120" left="0" top="0" /> 
9: <region id="Text" width="1 76" height="1 20" left="1 76" top ="0"/> 

10: <region id="foot" width="352" height="24" left="0" top="121" /> 

1 1 : </layout> 

12: <action id="act_movie" type="replace" dest="plugin://movie" region="Image" /> 
13: <action id="act_image" type="insert" dest="plugin://camera" region="lmage" /> 
14: <action id="act_text" type="insert" dest="plugin://textedit" region="Text" /> 
15: <action id="act_speech" type="insert" dest="plugin://speech coding" /> 
16: </head> 
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<body> 
<par> 
<seq> 
<par dur = "8s"> 
<video src = "Firstlmage.mpg" region="lmage" /> 
<text src = "FirstText.txt" region="Text" /> 
<audio src = "FirstSound.smf" /> 

<event ref="act_movie" src="UI" type="select" desc="MOVIE STARTUP"/> 
</par> 

<par dur = "7s"> 
<text src = "SecondText.txt" region="foot" /> 

<event ref="act_image" src="UI" type="select" desc="CAMERA STARTUP"/> 
<event ref="act_text" src="UI" type="select" desc="TEXT INPUT'7> 
<event ref^"act_speech" src="timer" type="2s" desc="SOUND RECORDING"/> 
</par> 

<par dur = "4s"> 
<img src = "Thirdlmage.jpg" region="Image"/> 
<text src = "ThirdText.txt" region="Text"/> 
<audio src = "ThirdSound.smf"/> 
</par> 
</seq> 

<text src = "BottomTexttxt" region="foot" begin="4s" end="l5s" /> 
</par> 
</body> 
</smil> 
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FIG.29 

1:<smil> 

2: <head> 

3: <meta name="title" content="navigation" /> 

4: <meta name="role" content="conterits" /> 

5: <layout> 

6: <root-layout width="352" height="144'7> 

7: <region id="top width="352" height="72" left="0" top="0" /> 

8: <region id="middle" width="352" height="48" left="0" top ="72"/> 

10: </layout> 

11: <action id="act_www" type="ext" dest="http://www.sharp.co.jp/sales" /> 

12: <action id="act_mail" type="ext" dest="mailto://sales@sharp.co.jp"/> 

13: <action id="act.phone" type="ext" dest="phone://09003476786" /> 

15: </head> 



FIG.30 

16: <body> 

17: <seq> 

18: <par dur = "10s"> 

19: <img src="ProductImage.jpg" region="top" /> 

20: <text src = "ProductDesc.txt" region="middle" /> 

21: <event begin="1s" dur="3s"ref="act_www" src="Ul" type="select" /> 

22: <event begin="4s" dur="3s" ref="act_phone" src="UI" type="select" /> 

23: <event begin="7s" dur="3s" ref="act_mail" src="Ul" type="select" /> 

24: </par> 

25: </seq> 

26: </body> 

27:</smil> 
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FIG.47 

1: <smil> 

2: <head> 

3: <meta name="title" content="navigation" /> 

4: <meta name="author" content="Okure" /> 

5: <meta name="role" content="template" /> 

6: <layout> 

7: <root-layout width="352" height=" 1 44"/> 

8: <region id="Image" width="1 76" height="120" left="0" top="0" /> 

9: <region id="Text" width=" 1 76" height=" 1 20" left=" 1 76" top ="0"/> 

10: </layout> 

11: </head> 

12: <body> 

13: <par> 

1 4: <seq> 

15: <par dur = "8s"> 

1 6: <img src = "Firstlmage.jpg" region="Image" /> 

17: </par> 

18: <par dur = "7s"> 

19: <img src = "SecondImage.jpg" region="lmage" /> 

20: <text src = "SecondText.txt" region="text" /> 

21: </par> 

22: <par dur = "4s"> 

23: <img src = "Thirdlmage.jpg" region="lmage"/> 

24: <text src = "ThirdText.txt" region="text" /> 

25: </par> 

26: </seq> 

27: </par> 

28: </body> 

29: </smil> 
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1:<smil> 
2: <head> 

3: <meta name="title" content="navigation" /> 

4: <meta name="author" content="Okure" /> 

5: <meta name="role" content="template" /> 

6: <layout> 

7: <root-layout width="352" height=" 1 44"/> 

8: <region id="Image" width="1 76" height="1 20" left="0" top="0" /> 

9: <region id="Text" width="l 76" height="l 20" left="l 76" top ="0"/> 

10: <region id="foot" width="352" height="24" left="0" top="121" /> 

11: </layout> 

12: <action id="act_movie" type="replace" dest="plugin://movie" region="Image" /> 

13: <action id="act_image" type="insert" dest="plugin://camera" region="lmage" /> 

14: <action id="act_text" type="insert" dest="plugin://textedit" region="Text" /> 

15: <action id="act_speech" type="insert" dest="plugin://speech_coding" /> 

16: </head> 
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FIG.54A 
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FIG.58 
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FIG.59 



CHECKING OF CHANGED 
DATA WITHIN SCENE 
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FIG.61 

17: <body> 

1 8: <seq> 

1 9: <par dur = "8s"> 

20: <img src = "Firstlmage.jpg" region="lmage" /> 

21: </par> 

22: <par dur = "7s"> 

23: <img src = "Secondlmagejpg" region="Image" /> 

24: <text src = "SecondTexttxt" region="Text" /> 

25: <event ref^'actjmage" src="UI" type="select" desc="CAMERA STARTUP"/> 

26: </par> 

27: <par dur = "4s"> 

28: <img src = "Thirdlmage.jpg" region="Image"/> 

29: <text src = "ThirdTexttxt" region="Text"/> 

30: <event ref^'acttext" src="Ul" type=" select" desc="TEXT INPUT'7> 

31: </par> 

32: </seq> 

33: </body> 

34:</smil> 
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FIG.63 
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FIG.67 



CALCULATION OF TIME OF 
RESUMPTION OF REPRODUCTION 
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FIG.68 

1:<smil> 

2: <head> 

3: <meta name="title" content="navigation" /> 

4: <meta name="readOnly" content="yes" /> 

5: <meta name="role" content="template" /> 

6: <layout> 

7: <root-layout width="352" height="144"/> 

8: <region id="Image" width="176" height="120" left="0" top="0" /> 

9: <region id="Text" width="1 76" height="1 20" left="1 76" top ="0"/> 

10: <region id="foot" width="352" height="24" left="0" top="121" /> 

11: </layout> 

12: <action id="act_movie" type="replace" dest="plugin://movie" region="lmage" /> 

1 3: <action id="act_image" type="insert" dest="plugin://camera" region="Image" /> 

14: <action id="act_text" type="insert" dest="plugin://textedit" region="Text" /> 

15: <action id="act_speech" type="insert" dest="plugin://speech_coding" /> 

16: </head> 
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FIG.69 



DETERMINATION OF PERMISSION 
FOR SLIDE EDITING 
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FIG.70 



1 :<smil> 
2:<head> 
3: <layout> 

4: <root-layout width="240" height="260"/> 

5: <region id="image" top="20" left="20" width="200" height="200" /> 
6: <region id="text" top="220" Ieft="l0" width="240" height="20" /> 
7: </layout> 
8:</head> 
9:<body> 
10: <par dur="2s"> 

11: <img src="top.jpg" region="image" /> 
1 2: <text src="top.txt" region="text" /> 
13: </par> 
14: <par dur="6s"> 

1 5: <img src="image.jpg" region="image" /> 

1 6: <audio src="bgm.amr" /> 

1 7: <text src="message.txt" region="text" /> 

18: </par> 

19:</body> 

20:</smil> 



FIG.71 

1:<html> 

2: <body> 

3: <form action="http://localhost/mmproc" method="post"> 

4: <label>SELECT PHOTO 

5: <input name="image.jpg" type="file" accept="image/*" /> 

6: </label> 

7: <label>ENTER BIRTHDAY MESSAGE 

8: <input name="message.txt" type="text" /> 

9: </label> 

1 0: <input type="submit" value="send"/> 

11: </form> 

12: </body> 

13:</html> 
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FIG.73 

1:<?xml version="1.0" encoding="UTF-8"?> 

2:<mmstemplate xmlns=http://www.foo.com/2003/MMSTemplate ...> 
3: <head> 

4: <title>Postcard Example</title> 

5: </head> 

6: <composition> 

7: <template src= ,/ http://vmw.example.com/postcard--comp.htm'V> 

8: </composition> 

9: <presentation> 
10: <template src='' http://www.example.com/postcard-rend.smiV> 
1 2: </ presentation> 
1 3:</mmstemplate> 
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FIG.75 

1:<?xml version="1.0" encoding="Shift_JIS"?> 
2:<smil> 

3:<head> 
4: <layout> 

5: <root-layout backgroundColor="black"/> 

6: <region id="main" top="0%" left="10%" width="80%" height="100%" z-index="1 "/> 
7: <region id="right" right="0%" top="0%" width="10%" height="100%" z-index="0'7> 
8: <region id="left" left="0%" top="0%" width="10%" height="100%" z-index="0"/> 
9: </layout> 
10:</head> 

1 1 :<body> 

12: <par repeatCount="indefinite"> 

13: <video id="tvcontents" src="data:,dummy" region="main" /> 

14: <img src=''http://www.example.com/tvinfo/r-info.jpg''region=''right" /> 

15: <img src="http://www.example.com/tvinfo/l-info.jpg"region="left" /> 

16: </par> 

1 7:</body> 

18:</smil> 



I II 3> 



FIG.76 

1:<?xml version="l.O" encoding="Shift_JIS"?> 

2:<psd version="l.O"> 

3: <channel>SMIL CHANNEL</channel> 

4: <day value="2003-09-29"> 

5: <embededid value= "tvcontents" /> 

6: <program> 

7: <schedule> 

8: <starttime value="0:00"/> 

9: <endtime value="2:00"/> 
10: </schedule> 
11: <title>ANIMATION</title> 
12: <ref href="ch_4"/> 

13: </program> 

1 4: <program> 

15: <schedule> 

1 6: <starttime value="2:00"/> 

17: <endtime value="4:00"/> 

1 8: </schedule> 

19: <title>SHAARP PRODUCT INFORMATION</title> 
20: <ref re^"www.example.com/product.smil"/> 

21: </program> 

(OMITTED: DESCRIPTION AT AND AFTER 4:00 CONTINUES) 

: </day> 

: <day value="2003-09-30"> 

: </day> 



:</psd> 



UJ§3 



FIG.77 

1:<?xml version="1.0" encoding="Shift_JIS"?> 

2:<mmstemplate> 

3: <head> 

4: <title>SMIL CHANNEL</title> 

5: </head> 

6: <composition> 

7: <template src="smil.psd"/> 

8: </composition> 

9: <presentation> 
10: <template src="tvlayout.smil"/> 
11: </ presentation> 
1 2:</ mmstemplate> 
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FIG.79 



Q START J 



SMIL IS PARSED 
STORED IN DAT 


) AND 

A STRUCTURE 


y 




«e 


REPRODUCTION FOLLOWING 
DATA STRUCTURE 



— S2401 



— S2402 



NO 



S2403 

HAS MEDIA 
DATA BEEN INSERTED 
OR SWITCHED? 



YES 



UPDATE OF VALUE TO WHICH 
DATA STRUCTURE CORRESPONDS 



S2405 



S2404 




.YES 



_^S2406 
COMPLETION OF ^" 
REPRODUCTION/TO TRANSMISSION 
OF E-MAIL? 



NO 




PREPARATION OF SMIL FROM —S2407 
DATA STRUCTURE 



I 



TRANSMISSION OR STORAGE 
PROCESSING 



i 

Q END J 



S2408 



FIG.80A 



DisplayScheduler 



java.util.LinkedList DisplayContentList 

// HOLDING OF DisplayContent CLASS IN UST 
int CurrentTime // TIME OF REPRODUCTION AT THIS TIME 



FIG.80B 



DisplayContent 



java.lang.String MediaType 


// 


MEDIA TYPE 


java.io.File MediaFile 


// 


FILE NAME 


java.awt.Rectangle Region 


// 


DISPLAY REGION 


int BeginTime 


// 


TIME OF START OF REPRODUCTION(sec) 


int EndTime 


// 


TIME OF COMPLETION OF REPRODUCTION 


java.lang.String EvtSrc 


// 


event src 


java.lang.String EvtType 


// 


event type 


java.lang.String EvtDesc 


// 


event desc 


java.lang.String Actld 


// 


action id 


java.lang.String ActType 


// 


action type 


java.net.URI ActDest 


// 


action dest 



it}/ 5 



93ft 





text 


BottomTexttxt 


CM" 

► CM LO Tj" 
O t— CO CM 




LO 


i 


i 


i 


i 


i 


i 


p 


audio 


ThirdSoundsmf 


o~ o~ O O 


LO 




i 


i 


i 


i 


i 


j 


o 


text 


ThirdTexttxt 


176, 
0, 

1 7fi 
I /O, 

120 


LO 


o> 


i 


i 


i 


i 


i 




CO 


img 


Thirdlmage.jpg 


CO o 
- P"- CM 
O O -r- 


LO 


CD 


i 


i 


i 


i 


i 


i 




audio 




cd o~ cT o 


00 


LO 


timer 


CO 
CM 


SOUND RECORDING 


act_speech 


insert 


plugin://speech_config 


CO 


text 




176 r 

176, 
120 


oo 


LO 




select 


TEXT INPUT 


-M 

X 
CD 

r- 

o 

CD 


insert 


plugin://textedit 


LO 


img 




CO o 
- P*- CM 

o o ^ ^ 


oo 




3 


select 


CAMERA STARTUP 


actjmage 


insert 


plugin://camera 




text 


SecondTexttxt 


i— ~ CM* 
. CM LO 
O CO CM 


CO 


LO 


i 


i 


i 


i 


i 


i 


00 


audio 


FirstSoundsmf 


o" o" o~ o 


o 


CO 


i 


i I 


i 


i 


i 


i 


CM 


text 


FirstText.txt 


176, 
0, 

176 
120 


o 


oo 




i 


i 


i 




i 




video 


Firstlmage.mpg 


CO o 

r-T r< ^ ^ 
O O f— i— 


o 


CO 




select 


MOVIE STARTUP 


actjnovie 


replace 


plugin://movie 


ATTRIBUTES 


Media 
Type 


Media 
File 


Media 

Region 

(x,y,w,h) 


Begin 
Time 


End 
Time 


EvSrc 


EvType I 


EvDesc 


Actld 


ActType 


ActDest 



00 

d 





text 


BottomTexttxt 


*— ~ cm" 

CM LO ^* 
O t- CO CM 




1 o 


i 


i 


i 


i 


i 


i 


o 


audio 


ThirdSound.smf 


o o" o" o 


LO 


a* 


1 


i 


i 


i 


i 


i 




text 


ThirdText.txt 


176, 
0, 

176 
120 

1 


LO 




i 


i 


i 


i 


i 


i 


CO 


bX) 

E 


Thirdlmagejpg 


CO o 
- P** CM 
O O i- ^ 


LO 


C75 


i 


i 


i 


i 


i 


i 




audio 


VoiceOl.amr 




o" o" o" o 


oo 


LO 


timer 


C/> 
CM 


SOUND RECORDING 


act_speech 


insert 


plugin://speech.config 


CO 


text 


EditText01.txt 




176, 
0, 

176, 
120 


CO 


LO 




select 


TEXT INPUT 


+-» 

X 
CD 

o 

CO 


insert 


p 1 ugin ://te xtedit 


LO 


img 


CameraPictOljpg 




CO o 
- p- cnj 

O O i- i- 


CO 


LO 


3 


select 


CAMERA STARTUP 


actjmage 


insert 


plugin://camera 




text 


SecondTexttxt 


^ CNJ 
. CM LO Tfr 
O CO CNJ 


00 


LO 


i 


i 


i 


i 


i 


i 


CO 


audio 


FirstSound.smf 


c> o~ o" o 


o 


oo 


i 


i 


i 


i 


i 


i 


CNJ 


text 


FirstText.txt 


176, 
0, 

176 
120 


o 


oo 


i 


i 


i 


i 


i 


i 




video 


Firstlmage.mpg 


CO O 

- cni 

O O t- 


o 


oo 


=3 


select 


MOVIE STARTUP 


actjnovie 


replace 


plugin://movie 


ATTRIBUTES 


Media 
Type 


Media 
File 


Media 

Region 

(x,y,w,h) 


Begin 
Time 


End 
Time 


EvSrc 


EvType 


EvDesc 


Actid 


ActType 


ActDest 



CM 
00 

O 



iO/532936 



FIG.83 

1:<smil> 

2: <head> 

3: <meta name="title" content="navigation" /> 

4: <meta name="author" content="Okure" /> 

5: <meta name="role" content="template" /> 

6: <layout> 

7: <root-layout width="352" height="144" /> 

8: <region id="lmage" width="l 76" height="120" left="0" top="0" /> 

9: <region id="Text" width="1 76" height="1 20" left="1 76" top="0" /> 

10: <region id="foot" width="352" height="24" left="0" top^l 21" /> 

1 1 : </layout> 

12: <action id="act_movie" type="replace" dest="plugin://movie" region= 

"Image" /> 

13: <action id="act_image" type="rep|ace" dest="plugin:// camera" region= 

"Image" /> 

14: <action id="act_text" type= "replace " dest="plugin://textedit" region="Text" /> 

15: <action id="act_speech" type= "replace " dest="plugin://speech_config" /> 

16: </head> 

17: <body> 

18: <par> 

19: <seq> 

20: <par dur = "8s"> 

21: <video src = "Firstlmage.mpg" region = "Image" /> 

22: <text src = "FirstText.txt" region = "Text" /> 

23: <audio src = "FirstSound.smf" /> 

24: <event ref ="act_movie" src="UI" type="select" desc="MOVIE STARTUP" /> 

25: </par> 

26: <par dur = "7s"> 

27: <text src = "SecondText.txt" region = "foot" /> 

<img src = "CameraPictOI jpg" region = "Image" /> 

<text src = "EditText01.txt" region = "Text" /> 

<audio src — Voiced .amr /y 

28: <event ref="act_image" src="Ul" type="select" desc="CAMERA STARTUP" /> 

29: <event ref="act_text" src="UI" type="select" desc="TEXT INPUT" /> 

30: <event ref="act_speech" src="timer" type="2s" desc="SOUND RECORDING" /> 

31: </par> 

32: <par dur = "4s"> 

33: <img src = "Thirdlmagejpg" region = "Image" /> 

34: <text src = "ThirdText.txt" region = "Text" /> 

35: <audio src = "ThirdSound.smf" /> 

36: </par> 

37: </seq> 

38: <text src="BottomText.txt" region="foot" begin="4s" end="15s" /> 

39: </par> 

40: </body> 
41: </smil> 
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FIG.84 



1:<smil> 
2: <head> 

3: <meta name="title" content="navigation" /> 
4: <meta name="author" content="Okure" /> 
5: <meta name="role" content="template" /> 
6: <layout> 

7: <root-layout width="352" height="144" /> 

8: <region id="Image" width="176" height="120" left="0" top="0" /> 
9: <region id="Text" width="l76" height="l20" Ieft="l76" top="0" /> 
<region id="foot" width="352" height="24" left="0" top="121" /> 
</layout> 
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25 
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29 
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31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 



</head> 
<body> 
<par> 
<seq> 
<par dur = "8s"> 
<video src = "Firstlmage.mpg" region = "Image" /> 
<text src = "FirstText.txt" region = "Text" /> 
<audio src = "FirstSound.smf" /> 

</par> 

<par dur = "7s"> 
<text src = "SecondText.txt" region = "foot" /> 
<img src = "CameraPictQ1.jpg" region = "Image" /> 



<text src = EditTextOI .txt region = Text /> 



<audio src = VoiceOLamr /> 



</par> 

<par dur = "4s"> 

<img src = "Thirdlmage.jpg" region = "Image" /> 

<text src = "ThirdText.txt" region = "Text" /> 

<audio src = "ThirdSound.smf" /> 
</par> 
</seq> 

<text src="BottomText.txt" region="foot" begin="4s" end="15s" /> 
</par> 
</body> 
</smil> 
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Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



